Hybrid tracking systems are known which enable somewhat accurate registration of virtual information, e.g., images, upon real objects in selected outdoor environments. Such outdoor applications often can be used in well-defined areas where it is feasible, e.g., to add a few objects to the environment that may serve to help the tracking. A better system utilizing such accurate registration is needed.
Often Augmented Reality systems employ a 6-D tracking system that can measure, in real time, the orientation and position of the user at all times. This may be required, e.g., for the system to know exactly how to draw the virtual objects so that they appear in their proper positions with respect to the real world in which the user is positioned and respecting which the user wishes to insert the virtual information, e.g., an image of a building on an empty lot.
Reasonably accurate tracking systems have also been developed for indoor settings, where the system designer may have much more control over the environment and may be more able to modify it as needed. However, the ability to track more accurately in outdoor environments, where the system designer may have no control over the environment, could enable more augmented reality applications. Currently, such satisfactory tracking has only been achieved under significantly constrained circumstances (e.g., the user does not walk around) or with prohibitively expensive equipment.
Fiducials, which are easily identifiable markers which can be added to the environment to make the visual tracking more simple, are known to be useful in position and orientation tracking. Use of fiducials in an augmented reality system is discussed, e.g., in Bajura, Mike and Ulrich Neumann, Dynamic Registration Correction in Augmented-Reality Systems. Proceedings of IEEE VRAIS '95 (Research Triangle Park, NC, Mar. 11–15, 1995), 189–196 (the disclosure of which is hereby incorporated by reference). Discussed therein, is an example of using fiducials to supplement 6-D tracking. A separate 6-D tracker, such as a magnetic-based system like the Polhemus, as discussed in Raab, F., Bood, E., Steiner, O., Jones. H. Magnetic position and orientation tracking system. IEEE Transactions on Aerospace and Electronic Systems, AES-15 (5), 1979, pp. 709–717, which generates an initial guess of the user's position and orientation. Then the fiducials which can be detected by a video tracking system can be used to correct this initial guess. The corrections can be applied to orientation only or in 6-D. Another example of this appears in State, Andrei, Gentaro Hirota, David T. Chen, Bill Garrett, and Mark Livingston, Superior Augmented Reality Registration by Integrating Landmark Tracking and Magnetic Tracking. Proceedings of SIGGRAPH '96 (New Orleans, La., Aug. 4–9, 1996), 429–438, and the related U.S. Pat. No. 6,064,749, issued to Hirota, et al. May 16, 2000, entitled HYBRID TRACKING FOR AUGMENTED REALITY (the disclosures of each of which are hereby incorporated by reference).
Other examples of visual-based tracking exist, which employ the detection of fiducials. These methods, however, do not use multiple sensors. Instead they typically find the 2-D locations of the fiducials in an image, and combining that with the known 3-D locations of the fiducials, they recover the 6-D location of, e.g., the camera creating the image. These methods suffer from the shortcoming, however, that they need to see, typically at a minimum, 3 fiducials at all times. In practice, however, such methods may not be stable unless they can include in the image a larger numbers of fiducials (e.g., 6 or more). For a narrow field of view camera this can mean the density of fiducials in the environment must be very large. These solutions tend to be more stable if the fiducials are spread widely apart (requiring multiple cameras or a wide field of view, but the wide field of view spreads the resolution across a large area). One example of pure fiducial tracking is discussed in Neumann, Ulrich and Youngkwan Cho, A Self-Tracking Augmented Reality System. Proceedings of VRST '96 (Hong Kong, Jul. 1–4, 1996), 109–115.